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“Out-of-field teaching” is an international phenomenon that seems particularly prevalent in 
mathematics. Our study is evaluating the impact of a national professional learning program 
for out-of-field secondary mathematics teachers in Ireland. Using the Productive Pedagogies 
framework, we compared the pedagogical practices of three pairs of teachers who were either 
upskilled, still out-of-field, or always in-field. The findings suggest that graduates of the 
upskilling program are developing pedagogical practices more like those of in-field teachers. 


“Out-of-field” teaching is an international phenomenon that involves teachers being 
assigned to teach subjects that do not match their training or education (Ingersoll, 2002). 
This practice seems particularly prevalent in the teaching of mathematics. Out-of-field 
teachers of mathematics typically possess a teaching qualification but have limited advanced 
studies of mathematics and little or no specific preparation in mathematics pedagogy. There 
is growing recognition of the need for professional development programs that meet the 
particular needs of out-of-field teachers (Du Plessis et al., 2014). To date, however, there 
has been little research on the effectiveness of such programs (Faulkner et al., 2019). This 
paper reports on aspects of a larger study that is evaluating the impact of a long-term, large- 
scale, government-funded, nationally-consistent, university-accredited program offered to 
out-of-field teachers of mathematics in Ireland — the Professional Diploma in Mathematics 
for Teaching (PDMT). 


Background to the Study 


In Ireland, concerns about student performance in post-primary school mathematics at 
the beginning of the 21“ century led to the introduction in 2010 of a new curriculum that 
shifted emphasis towards understanding and problem-solving and away from memorisation 
and procedures (National Council for Curriculum and Assessment, 2005). Concurrently, the 
Teaching Council of Ireland (2013) introduced new accreditation requirements for initial 
teacher education programs. In mathematics, fully qualified teachers must have a degree- 
level qualification with the specific study of mathematics comprising at least one-third of 
the degree. There are also minimum credit requirements in analysis, algebra, geometry, and 
probability and statistics, with additional credits to be obtained in a variety of optional topics. 
Despite these strict requirements, school principals in Ireland have autonomy in recruiting 
staff and assigning teachers to subjects and classes, thus leaving open the possibility of 
placing teachers in out-of-field positions. 

Ni Riordain and Hannigan (2009) speculated that the phenomenon of out-of-field 
teaching of mathematics could be a possible obstacle to achieving the goals of the new 
mathematics curriculum. They conducted a national survey of teachers of mathematics in 
Irish post-primary schools, collecting data on respondents’ teaching assignments, degree 
qualifications, and the subjects they were qualified to teach according to the requirements 
specified by the Irish Teaching Council. This survey established that 48% of respondents 
were teaching mathematics without the necessary subject-specific qualifications. In response 
to this finding, the Department of Education and Skills (DES) funded the PDMT to develop 
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the content and pedagogical content knowledge of out-of-field teachers of mathematics to 
the level required by the Teaching Council. Six cohorts comprising 1078 teachers 
participated in the PDMT from 2012-2020. 

The PDMT is a 2-year part-time postgraduate program with teachers’ tuition fees funded 
by the DES. Delivery of the program is led by the University of Limerick in conjunction 
with a national consortium of higher education institutions. PDMT participants teach full- 
time in their schools while they undertake the program in the evening, on weekends, and 
during school vacations via a blended learning approach. Ten undergraduate mathematics 
modules are delivered online in 30-hour blocks across 6-week sessions, with additional face- 
to-face and online support. Two yearlong mathematics pedagogy modules are delivered 
face-to-face via weekend workshops and a one-week summer school. These pedagogy 
modules emphasise classroom practices that support problem-solving and promote 
conceptual understanding. One of the pedagogy modules also requires participants to 
complete a supervised action research project on their practice in the mathematics classroom. 

An important aim of the PDMT is to develop out-of-field teachers’ knowledge of 
mathematics content and pedagogy. The program additionally aims to support teachers in 
developing pedagogical practices aligned with the goals of the new mathematics curriculum 
in Ireland, and this is the focus of the present paper. To gain insights into the latter aspect of 
the PDMT, we compared video-recorded mathematics lessons taught by teachers who were 
currently, formerly, or never out-of-field in order to address the following research question: 
What similarities and differences can be observed when comparing the pedagogical 
practices of out-of field, upskilled, and in-field teachers of mathematics? (Upskilled teachers 
are those who have completed the PDMT.) 


Conceptualising and Evidencing the Impact of Professional Development 


Researching the impact of teacher professional development poses methodological and 
conceptual challenges. Desimone (2009) discussed the strengths, weaknesses, and trade-offs 
between observations, interviews, and surveys as the most common methods for studying 
teacher learning, and stressed the importance of choosing data collection methods to match 
a study’s research questions. Adler et al. (2005) also pointed out that a personal investment 
in teaching makes it difficult for teacher educators to take a critical stance towards the 
research we do with teachers, and they suggested developing strong theoretical languages in 
order to distance ourselves from what we are looking at. In the present study, as the authors 
have the dual roles of researchers and teacher educators in the PDMT, we aimed to achieve 
this critical distance by situating our research within Desimone’s (2009) conceptual 
framework for studying teacher professional development. 

Desimone’s (2009) framework has two components. The first component identifies the 
critical features that define effective professional development in terms of increasing teacher 
knowledge and skills and improving their practice. Drawing on existing empirical research, 
Desimone proposed that this set of critical features places emphasis on: (a) content focus, 
(b) active learning, (c) coherence, (d) duration, and (e) collective participation. The second 
component of the conceptual framework is “an operational theory of how professional 
development works to influence teacher and student outcomes” (p. 184). For this component, 
Desimone proposed a model with the following steps: 

1. Teachers experience effective professional development (defined in terms of the set 

of critical features outlined above). 

2. The professional development increases teachers’ knowledge and skills and/or 

changes their attitudes and beliefs. 
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3. Teachers use their new knowledge and skills, attitudes, and beliefs to improve the 

content of their instruction or their approach to pedagogy, or both. 

4. The instructional changes foster increased student learning. (p. 184) 

Desimone (2009) acknowledged that other potentially important factors existed, but 
these were not incorporated into her model because they have not yet been the subject of 
much research on the impact of professional development. These factors might include, for 
example, professional identity (Hobbs, 2012), the role of the principal in providing 
opportunities for teacher learning (Du Plessis et al., 2015), and the role of curriculum 
materials and implementation (Remillard & Heck, 2014). Desimone also conceded that her 
model could be criticised as representing a positivist viewpoint. However, she maintained 
that the model could still be used in studies with different theoretical perspectives on teacher 
learning as a means of integrating the knowledge generated by empirical research with “the 
emerging consensus of what is good professional development” (p. 187). 

Desimone (2009) noted that it is rare for a single study to investigate all four elements 
of her proposed model; in particular, there are significant methodological difficulties in 
designing evaluations that measure the effects of professional development on student 
achievement. Research conducted by our larger team has analysed the critical features of the 
PDMT program (Step 1 in Desimone’s model; see Goos et al., 2020) and its effect on the 
teachers who participated in the program (Steps 2 and 3; see Lane & Ni Riordain, 2020; Ni 
Riordain et al., 2017). In this paper, we further examine the impact of the PDMT on teachers’ 
pedagogical practices (Step 3) as a key element in Desimone’s model of teacher change. 


Research Design and Methods 


We would have liked to investigate the effects of the PDMT on participants’ classroom 
teaching approaches by observing lessons taught before and after the teachers experienced 
the program. However, this was not possible due to resource constraints and the demands of 
delivering a large, complex program involving 13 higher education institutions. Our research 
team’s earlier analysis of PDMT participants’ action research reports indicated that teachers 
perceived a shift in their pedagogical practices towards more student-centred approaches that 
emphasised conceptual understanding and problem-solving (Lane & Ni Riordain, 2019). To 
further investigate these teacher self-reports, we designed a cross-sectional study to compare 
the pedagogical practices of three groups of teachers: (a) those currently teaching 
mathematics out-of-field (n=2); (b) those who had been upskilled to fully qualified status by 
completing the PDMT (n=2); and (c) those who had always been fully qualified, in-field 
teachers of mathematics (n=2). These six teachers were recruited from six different schools. 

Teachers were observed by the second author as they taught six junior secondary 
mathematics lessons in two blocks of three consecutive lessons. These lessons were also 
video-recorded for later analysis. Pre- and post-lesson interviews were conducted by the 
second author to obtain teachers’ perspectives on lesson objectives, anticipated and actual 
challenges or successes, knowledge, and confidence levels. Surveys were also administered 
to the teachers to collect demographic information and data on teacher self-efficacy, job 
satisfaction, and preparedness for teaching topics in the secondary mathematics curriculum. 
All data collection was carried out by the second author. This paper draws only on teacher 
demographic data and the video recordings of lessons they taught. 

The Productive Pedagogies framework was selected as a classroom observation 
instrument that has been theoretically and statistically validated in Australian research 
(Lingard et al., 2001). Although not specifically designed for mathematics classrooms, it has 
been used in longitudinal studies of mathematics teaching (e.g., Makar, 2011) as well as in 


205 


Goos and Guerin 


large-scale studies of primary and secondary school lessons in a range of curriculum areas. 
The 20 items of the Productive Pedagogies framework are shown in Figure 1. The framework 
has four dimensions, two concerned with the academic outcomes of schooling (left side of 
Figure 1) and two with the social outcomes (right side of Figure 1). The Intellectual Quality 
dimension emphasises the importance of all students being presented with challenging work. 
Connectedness makes learning meaningful by linking new knowledge to prior knowledge, 
other subjects in the curriculum, and the world beyond school. Supportive Classroom 
Environment foregrounds relationships and giving students a voice in the classroom, while 
Recognition of Difference provides students with the capacity to act as responsible members 
of a democratic society. A 5-point rating scale is used to provide an index of the variation in 
quality of classroom practice for each item. 


Intellectual Quality Supportive Classroom Environment 

Higher order thinking (HOT) Student direction (SD) 

Deep knowledge (DK) Social support (SS) 

Deep understanding (DU) Academic engagement (AE) 

Substantive conversation (SC) Explicit quality performance criteria (EC) 
Problematic knowledge (PK) Student self-regulation (SS) 
Meta-language (ML) 

Connectedness Recognition of Difference 
Knowledge integration (KI) Cultural knowledge (CK) 
Background knowledge (BK) Inclusivity (1) 

Problem-based curriculum (PBC) Narrative (N) 


Connectedness beyond the classroom (CBC) Group identities (GI) 
Active citizenship (AC) 


Figure 1. Productive Pedagogies dimensions. 


Before observing and video-recording lessons taught by the six teachers, the second 
author discussed the Productive Pedagogies scoring manual with the first author, who is an 
experienced user of the Productive Pedagogies framework. Both authors used the scoring 
manual independently to rate an online video of a junior secondary mathematics lesson, after 
which they compared their ratings and resolved any differences via further discussion. After 
the data collection was completed, the second author watched the video-recorded lessons, 
assigned scores for each item, and calculated mean scores on each dimension for each of the 
three types of teachers (out-of-field, upskilled, in-field). Similarities and differences between 
the teachers were further examined for each dimension by inspecting item scores. 


Results 


Demographic Data 


Table 1 summarises the gender, years of mathematics teaching experience, and grouping 
(out-of-field, upskilled, in-field) of the participating teachers. Both out-of-field teachers 
were female and had taught mathematics for up to 10 years; the other teachers were male 
with mathematics teaching experience ranging from less than five to more than 16 years. 
Table 1 also shows the year in which upskilled and in-field teachers gained their mathematics 
teaching qualifications through the PDMT or initial teacher education program respectively. 
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Table | 
Teacher Demographic Characteristics 
Teacher 
Characteristic Tl TO T3 T4 T5 T6 
Gender/ Group M M F M F M 
US IF OOF US OOF IF 

Years teaching mathematics 16-20 = 11-15 <5 <5 6-10 6-10 
(year qualified) (2018) (1999) (n/a) (2018) (n/a) (2010) 


Note. OOF = out-of-field; US = upskilled; IF = in-field 


Pedagogical Practices 


Table 2 presents the mean scores on the Productive Pedagogies dimensions for each 
group of teachers over the three lessons for which they were observed. Thus, each mean 
score is derived from six observations (two teachers x three lessons). One observable trend 
is that out-of-field, upskilled, and in-field teachers all scored highest on the dimension of 
Supportive Classroom Environment and lowest on the dimension of Connectedness. The 
same pattern was found in Makar’s (2011) analysis of pedagogical practices in Australian 
primary school teachers’ “regular” mathematics lessons. 


Table 2 
Productive Pedagogies Mean Scores 


Teacher Group 
Dimension Out-of-Field  Upskilled In-Field 
Intellectual Quality 2.64 3.00 3.61 
Connectedness 1.54 1.79 1.75 
Supportive Classroom Environment 3.67 327 4.07 
Recognition of Difference 3.10 2.2 2.57 


Note. A 5-point rating scale was used. Each group comprises two teachers who were observed for three lessons. 


Looking across the rows of Table 2 enables comparison between the three groups of 
teachers on each Productive Pedagogies dimension. In-field teachers had the highest mean 
scores for the dimensions of Intellectual Quality and Supportive Classroom Environment, 
while upskilled teachers recorded the highest mean score for Connectedness — although this 
was very similar to the mean score of the in-field teachers. Out-of-field teachers achieved 
the highest mean score for the dimension of Recognition of Difference. This may be because 
they were the only teachers in the sample who taught mixed-ability, rather than streamed, 
mathematics classes. These two teachers were observed to place particular emphasis on 
encouraging participation of struggling students, thus highlighting the element of Inclusivity 
(Figure 1) for this non-dominant group in their classrooms. 

Because the PDMT is mainly concerned with teaching mathematics for academic 
outcomes, we next examine the detail of teachers’ pedagogical practices in the corresponding 
dimensions of Intellectual Quality and Connectedness. Tables 3 and 4 show each teacher’s 
score totals for the three observed lessons for each item of these dimensions. (Score totals 
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are displayed instead of mean scores for ease of comparison across multiple teachers and 
items.) Pedagogical practices that seem to characterise the greatest difference between 
teacher groups are highlighted for discussion. 


Table 3 
Intellectual Quality Score Totals 


Out-of-Field Upskilled In-Field 
Item tc) 15 T1 T4 T2 T6 
Higher Order Thinking 8 8 : 10 8 15 
Deep Knowledge 9 11 i ibe ies) 
Deep Understanding 9 12 10 10 12 {2 
Substantive Conversation 5 9 5 10 8 9 
Problematic Knowledge 6 6 5) 8 ia 12 
Meta-language 2) 9 9 8 12 2) 


Note. A 5-point rating scale was used. Each teacher was observed for three lessons. 


Within the dimension of Intellectual Quality, the greatest differences — equivalent to at 
least 6 points across three lessons, or a mean of 2 points per lesson on the 5-point observation 
scale — occurred on the items representing Higher Order Thinking, Deep Knowledge, and 
Problematic Knowledge (Table 3). The general trend is for the scores to increase from out- 
of-field to upskilled to in-field teachers. Also notable is the high Meta-language score for in- 
field teacher T2, who regularly provided help in the use of mathematical terminology for 
students who had been identified with low literacy skills. 

Figure 2 provides examples of questions posed by Teacher 5 (out-of-field), Teacher 4 
(upskilled) and Teacher 6 (in-field) that illustrate differences in the quality of their 
pedagogies for promoting Higher Order Thinking. According to the Productive Pedagogies 
classroom observation manual, Higher Order Thinking requires students to manipulate 
information and ideas in ways that transform their meaning and implications, for example 
by synthesising, generalising, explaining, or arriving at a conclusion or interpretation. This 
level of thinking is evident in the question asked by Teacher 6, and to some extent by Teacher 
4. However, Teacher 5’s question only requires students to rehearse procedural routines. 


T5 (OOF) T4 (US) T6 (IF) 
Solving equations Simultaneous equations Introducing simultaneous equations 


Q. Now what happens if I Q. How do I get this—3x | Q. Which of these equations do you think is 


have the scales andI take 8 | to become an x? the hardest to solve? Why? 
away from 12 on the RHS? 1) 95x* —2x+105=0 
Q. Can you explain to me 2) 3x+2y=8 
Q. What did she do to both | what you did? 3) 9x* — 39x? + 9x? — 90x + 3035 =0 
sides of the equation? 4) V9x4 a 45X _ 97x =0 


Figure 2. Examples of teacher questions illustrating variation in promotion of Higher Order Thinking. 


For the dimension of Connectedness, the differences between teacher groups were less 
pronounced — perhaps as a consequence of the lower scores across all three groups (see Table 
2). The greatest difference — equivalent to at least 3 points across three lessons, or a mean of 
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1 point per lesson on the 5-point observation scale — occurred on the item representing 
Problem-Based Curriculum (Table 4). In line with the Intellectual Quality dimension, the 
trend here is for scores to increase from out-of-field to upskilled to in-field teachers. 


Table 4 
Connectedness Score Totals 


Out-of-Field Upskilled In-Field 
Item 13: 15 T1 T4 T2 16 
Knowledge Integration 5 3 4 3 3 3 
Background Knowledge 6 7 i} 6 6 6 
Problem-Based Curriculum 6 6 af ° 8 10 
Connectedness Beyond the 3 3 4 3 3 3 


Classroom 


Note. A 5-point rating scale was used. Each teacher was observed for three lessons. 


Figure 3 shows examples of tasks presented by Teacher 3 (out-of-field), Teacher 1 
(upskilled) and Teacher 2 (in-field) that illustrate differences in the quality of their 
pedagogies for promoting a Problem-Based Curriculum. The Productive Pedagogies 
classroom observation manual defines a problem as a task with no specified correct solution 
that requires knowledge construction on the part of students. In keeping with the 
mathematics education research literature, we re-interpreted this definition to mean that a 
mathematical problem is a task for which the student does not know, and needs to construct, 
the solution method (National Council of Teachers of Mathematics, 2000). There is some 
evidence that this kind of knowledge construction 1s called for in the tasks offered by Teacher 
2 and Teacher 1; however, the task set by Teacher 3 instead requires using well-defined 
algorithms for algebraic manipulation. 


T3 (OOF) T1 (US) T2 (IF) 
Expanding and simplifying Introducing patterns Introducing Pythagoras’ Theorem 
4(x + 2) X, Y, Z, X, Y, Z, X,Y, Z, --- (Counting up boxes in the squares of the 
6(a + 4) + 2(2a + 3) What letter is in the 63"4 sides) ... How many boxes should be in 
position? here (the square on the hypotenuse) based 
on what we did earlier? 


Figure 3. Examples of tasks illustrating variation in problem-based lessons 


Conclusion 


In this paper, our focus was on the extent to which the PDMT encouraged teachers to 
take up pedagogical practices that emphasise conceptual understanding and problem- 
solving, in line with Ireland’s new secondary mathematics curriculum. Because it was not 
possible to collect longitudinal data on PDMT participants, we instead designed a cross- 
sectional study to identify similarities and differences between these upskilled teachers and 
other teachers of mathematics who were still out-of-field or had always been in-field. This 
design does not allow us to make claims about causality in relation to the PDMT, but it does 
illuminate some interesting comparisons between these three groups of teachers. The groups 
were similar in that out-of-field, upskilled, and in-field teachers all scored highest on 
Supportive Classroom Environment and lowest on Connectedness, a finding that aligns with 
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previous research using the Productive Pedagogies protocol (Makar, 2011). Some of the 
differences between groups suggested that upskilled teachers (PDMT graduates) might be 
adopting pedagogical practices more like those of in-field teachers than those who are still 
teaching mathematics out-of-field, especially in relation to promoting Intellectual Quality 
and Connectedness. These conclusions can only be tentative, given the small sample, but 
they suggest that structured lesson observations can usefully supplement upskilled teachers’ 
self-reports of changes in their pedagogical practices arising from participation in a targeted 
professional development program. In addition, such structured lesson observations may be 
useful for informing the design of programs to develop out-of-field teachers’ (and also pre- 
service teachers’) knowledge of mathematics and pedagogical practices, particularly in 
pinpointing specific items within the academic outcomes of schooling that require further 
consideration (e.g., knowledge integration and connectedness beyond the classroom). 
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